Osogbo
Unveiling Cultural Blind Spots: Analyzing the Limitations of mLLMs in Procedural Text Comprehension
Yari, Amir Hossein, Koto, Fajri
Despite the impressive performance of multilingual large language models (mLLMs) in various natural language processing tasks, their ability to understand procedural texts, particularly those with culture-specific content, remains largely unexplored. Texts describing cultural procedures, including rituals, traditional craftsmanship, and social etiquette, require an inherent understanding of cultural context, presenting a significant challenge for mLLMs. In this work, we introduce CAPTex, a benchmark designed to evaluate mLLMs' ability to process and reason about culturally diverse procedural texts across multiple languages using various methodologies to assess their performance. Our findings indicate that (1) mLLMs face difficulties with culturally contextualized procedural texts, showing notable performance declines in low-resource languages, (2) model performance fluctuates across cultural domains, with some areas presenting greater difficulties, and (3) language models exhibit better performance on multiple-choice tasks within conversational frameworks compared to direct questioning. These results underscore the current limitations of mLLMs in handling culturally nuanced procedural texts and highlight the need for culturally aware benchmarks like CAPTex to enhance their adaptability and comprehension across diverse linguistic and cultural landscapes.
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- Asia > Middle East > Iran (0.04)
- Asia > Japan (0.04)
- (17 more...)
- Leisure & Entertainment (1.00)
- Education (1.00)
Knowledge of artificial intelligence must be domesticated – Experts
Experts have suggested that the knowledge of Artificial Intelligence must be domesticated in Nigeria for the nation to meet up with the world. The experts explained that government, academia, community, and private sector must come together to rejuvenate Artificial Intelligence knowledge. This was disclosed during an annual lecture by the College of Science, Engineering and Technology, Osun State University, in honour of its pioneer Provost, Prof'Diran Famurewa held at the institution's Auditorium in Osogbo on Thursday. The Vice-Chancellor of Summit University Offa, Kwara, Prof. Abiodun Musa, professor of Mechatronics and who was the guest lecturer said Nigerian universities must not only generate money, but they must generate knowledge to solve community problems and needs as students must learn to solve community problems. The Vice-Chancellor, who is also an expert in Artificial Intelligence and Robotics, explained that if Nigeria needs to go beyond user to developer, it needs to rejuvenate artificial intelligence by looking into curriculum and implementation.
- North America > United States (0.27)
- Africa > Nigeria > Osun State > Osogbo (0.27)
- Europe > Ireland (0.07)